STEP 5

Evaluate your target interventions and results

Evaluation methods

Different designs

Different evaluation designs can be used. The experimental design (RCT) is by far the ideal method to assess the benefits of any given intervention. However, this design does not apply to practices in the real world. Furthermore, the RCT design does not allow for any adjustments that might have to be implemented in your institution.

There is a multiplicity of worthwhile design options available to assess the impact of your interventions. DMAA Outcomes Guide ¹lists 12 different 'study design characteristics':

randomized controlled trial (RCT)
pre-post with the patient as their own control
pre-post historical control design with concurrent control
pre-post historical control with actuarial adjustment based savings analysis
pre-post historical control with no actuarial adjustment trend of the non-diseased population
pre-post multiple baseline (with randomized controls) available comparison group
benchmark study
pre-post time series
post-only control group
post-only/no control group
cross-sectional (no comparison)
simple time-series (e.g. run charts).

Multiple options... and limitations

As such, the RCT design is not feasible or suitable for implementation in the field. Multiple design options are available, but all of them have limitations. One important principle is to do one's best to make it possible for comparisons to be made with referent groups. In addition, confounding factors should be assessed and will have to be taken into consideration when analyzing and interpreting results.

Collecting data

There are different ways of collecting data. Our choice will be guided by what questions we are trying to answer and what outcomes we want to measure. Similarly, there will be advantages and disadvantages to using any data collection method.

[1] Outcomes Consolidation Steering Committee. Disease management program guide: including principles for assessing disease management programs. Washington, DC: DMAA, 2005: 33.